fix(reporters): unwrap TestFailedException for failure grouping#5776
fix(reporters): unwrap TestFailedException for failure grouping#5776
Conversation
GitHub/HTML/JUnit reporters were grouping and labelling failures by the TestFailedException wrapper type instead of the real cause, so every failed test rolled up as "TestFailedException" in the Quick diagnosis and Failures by Cause sections. The wrapper exists to filter TUnit-internal frames from console stack traces (added in SimplifyStacktrace when stdout is a console). Keep that behaviour, but expose the original exception via a new WrappedException property and a static Unwrap helper, then unwrap at the reporter boundary so type detection sees e.g. InvalidOperationException. Unwrap loops to handle wrapper-of-wrapper and is null-tolerant via NotNullIfNotNull.
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Code Review
This is a clean, well-scoped fix for a real UX problem. The root cause (reporters reading type/message from the TUnitFailedException wrapper rather than the original exception) is clearly diagnosed, and the solution is appropriately targeted.
What's done well
- The
whileloop inUnwrap()correctly handles potential chains of wrappers (future-proof). [NotNullIfNotNull]is applied correctly — the static signature accurately communicates the null-preservation guarantee.- The test
AfterRunAsync_Unwraps_TestFailedException_For_Groupingdirectly validates the original bug (grouping key and Quick Diagnosis both show the real type), and the existing 14 reporter tests confirm no regression. - The split between MTP's console reporter (keeps the wrapper for filtered stack traces) and the file reporters (unwrap to the real exception) is a sound design decision that's explained in the PR summary.
Issues worth discussing
1. Stack trace behavior change in JUnit/HTML (medium)
In JUnitXmlWriter.WriteFailure and WriteError, the writer calls exception.ToString() after unwrapping:
var exception = TUnitFailedException.Unwrap(failed.Exception);
// ...
writer.WriteString(SanitizeForXml(exception.ToString()));exception.ToString() on the original exception includes its full, unfiltered stack trace — including TUnit-internal frames that TUnitFailedException.FilterStackTrace was specifically stripping. Previously these reporters saw only the filtered trace; now they see the raw one.
For CI tooling (GitLab/Jenkins consuming JUnit XML) and HTML reports, having the full stack trace is arguably better — developers drilling into a report want all the frames. But this is a silent behavior change worth explicitly acknowledging in the PR description or a comment near the unwrap call, since the original intent of FilterStackTrace was to reduce noise.
2. WrappedException naming could be clearer
WrappedException is grammatically ambiguous — it could mean "the exception that is wrapped" or "the exception that wraps something". The held value is the original exception that TUnitFailedException encapsulates, so OriginalException or InnerException (modulo the existing InnerException semantics) would remove any guesswork. Minor, but API names on a public abstract class matter.
3. Test coverage is incomplete for HTML and JUnit reporters
The new test validates GitHubReporter grouping, but HtmlReporter.MapException and JUnitXmlWriter.WriteFailure/WriteError have no dedicated tests for the unwrap behavior. If someone later refactors those code paths and drops the Unwrap call, there's no safety net. A couple of focused unit tests for those writers (similar in style to the new GitHubReporterTests test) would close this gap.
4. Hardcoded fallback type in WriteFailure/WriteError is now misleading (pre-existing, surfaced by this PR)
// When exception == null after Unwrap:
writer.WriteAttributeString("type", "TestFailedException");Before this PR, a null exception was rare (only when the property had no exception). That path now carries an inaccurate type literal — if failed.Exception is null, attributing it to "TestFailedException" is wrong. The fallback would read better as "UnknownException" or just "Unknown". This predates the PR but is now more visible.
Summary
The core fix is correct and the approach is sound. The main things worth addressing before merge:
- Decide whether the unfiltered stack trace in JUnit/HTML output is intentional, and document it if so.
- Add at least minimal tests for
HtmlReporterandJUnitXmlWriterunwrap paths. - Consider renaming
WrappedException→OriginalExceptionfor clarity (optional, given it's on a public type).
Summary
TestFailedExceptionbecauseSimplifyStacktracewraps the real exception in aTestFailedExceptionto strip TUnit-internal frames from the console stack trace. Quick diagnosis and Failures by Cause both showed the wrapper type instead of e.g.InvalidOperationException.WrappedExceptionproperty + staticUnwraphelper onTUnitFailedException.Unwrapis null-tolerant ([NotNullIfNotNull]) and loops to handle wrapper-of-wrapper.GitHubReporter(GetExceptionTypeName,GetError),HtmlReporter.MapException, andJUnitXmlWriter(WriteFailure,WriteError) now unwrap before reading type / message / stack. MTP's built-in console reporter still receives the wrapper, so filtered stack traces are preserved.Test plan
AfterRunAsync_Unwraps_TestFailedException_For_Groupingverifies grouping and Quick diagnosis use the inner exception type.GitHubReporterTestspass.StackTraceFilterTestspass (no regression in wrapper behaviour).TUnit.Enginebuilds clean acrossnetstandard2.0;net8.0;net9.0;net10.0.